Skip to content

PC Explorer visualization (damage classification COG) - #175

Merged
Anthony Cintron (prbatero) merged 9 commits into
mainfrom
prbatero/feat/data-publishing-pc-explorer
Sep 1, 2026
Merged

PC Explorer visualization (damage classification COG)#175
Anthony Cintron (prbatero) merged 9 commits into
mainfrom
prbatero/feat/data-publishing-pc-explorer

Conversation

@prbatero

Copy link
Copy Markdown
Contributor

Stacked on #123 (data-publishing-pc). Base is prbatero/feat/data-publishing-pc, not main — review the 6 explorer commits. Retarget to main once #123 merges.

Why

PC-published collections were created and listed, but had no Launch in Explorer and didn't appear in the Explorer's dataset picker. The Explorer requires a render configuration (+ mosaic + tile settings), and its renderer is TiTiler raster-only — our published assets are vector (GeoPackage/GeoJSON), so there was nothing to draw.

What

Render a licensing-safe view: rasterize our own damage output into a single-band classification COG (no source imagery pixels), publish it as a renderable item asset, and register the config the Explorer needs.

  • Phase 1raster.py: rasterize the damage GeoPackage → uint8 COG (1=damaged, 0=undamaged, 255=nodata), metric UTM grid clipped to AOI, per-side pixel cap with coarsening, valid COG (overviews). Reuses tile.py's detect_damage_mask so it agrees with the thumbnail.
  • Phase 2 — publish the COG as a damage_class item asset (+ collection item_assets), staged under published/<id>/; cleaned up by the existing finalize_unpublish.
  • Phase 3 — transport: render-options / mosaics / tile-settings methods.
  • Phase 4 — after item publish, register a damage raster-tile render option (red/grey colormap, transparent nodata), a most-recent mosaic, and tile-settings. Idempotent; gated on a staged COG so it never points at a missing asset.
  • Phase 5 — config knobs (PUBLISH_EXPLORER_RENDER_ENABLED default on, raster meters/max-pixels/min-zoom).

Design: spec/features/data-publishing/explorer-visualization.md.

Safety

  • No source pixels published — only the derived damage classification, carrying the same license/attribution as the vector output.
  • All new work is best-effort: rasterization or config-registration failure never fails the publish.
  • PC target only; Local unaffected.

Tests

New test_raster.py (rasterization + COG) and provider/transport tests (asset injection, idempotent config registration, cleanup). Full publishing suite green (169). Env-drift check green.

🤖 Generated with Claude Code

@prbatero
Anthony Cintron (prbatero) requested a review from a team as a code owner August 29, 2026 12:18
Base automatically changed from prbatero/feat/data-publishing-pc to main August 29, 2026 14:03
@prbatero
Anthony Cintron (prbatero) force-pushed the prbatero/feat/data-publishing-pc-explorer branch from f7013f5 to a6eec4c Compare August 29, 2026 14:06
@github-actions

Copy link
Copy Markdown

RC artifacts ready

All branch deployment references use the same RC tag:

  • hastegeo_version: 1.0.39rc1
  • training_image_tag: 1.0.39rc1
  • imageprep_image_tag: 1.0.39rc1
  • wheel: hastegeo-1.0.39rc1-py3-none-any.whl

Stacked feature (-pc-explorer): make PC published collections explorable
by rendering a licensing-safe damage classification COG (our derived
output, no source pixels) plus the render-option/mosaic/tile-settings
configuration the Explorer requires.
Phase 1. New publishing/raster.py rasterizes the published damage
GeoPackage into a single-band uint8 COG (1=damaged, 0=undamaged,
255=nodata), reusing tile.py's detect_damage_mask so it agrees with the
collection thumbnail. Projects to a metric UTM grid clipped to the AOI,
coarsens resolution to stay within a per-side pixel cap, and writes a
valid COG (GTiff-in-memory -> GDAL COG create-copy with overviews).
This is HASTE's derived output, so it carries no source imagery pixels.
Phase 2. At PC publish time the provider rasterizes the damage output to
a COG (idempotent per dataset), stages it under published/<id>/, and
injects a 'damage_class' asset (role data) into the item plus a matching
item_assets entry on the collection -- the renderable raster the Explorer
render config will point at. Shares the buildings/AOI loader with the
thumbnail and the finalize_unpublish cleanup (now also runs when Explorer
rendering is on, since a COG is written even without a dedicated store).
Best-effort: rasterization failure never fails the publish.
Phase 3. Add get/create render-options, get/create mosaics, and replace
tile-settings on the GeoCatalog REST adapter (the .../configurations/*
endpoints the Explorer requires). GETs unwrap list/dict-wrapped bodies
and treat 404 as empty so the provider can ensure config idempotently.
Phase 4. After the item is published (alongside the thumbnail), register
the Explorer visualization config idempotently: a 'damage' raster-tile
render option (colormap 1=red/0=grey, nodata transparent, pointing at the
damage_class asset), a 'most-recent' mosaic, and tile-settings. Gated on a
staged damage COG so the render option always references a real asset;
create-if-absent so a second dataset to the same collection doesn't
duplicate. Best-effort: never fails the publish.
Phase 5. Wire PUBLISH_EXPLORER_RENDER_ENABLED (default on),
PUBLISH_DAMAGE_RASTER_METERS (0.5), PUBLISH_DAMAGE_RASTER_MAX_PIXELS
(8192), and PUBLISH_DAMAGE_RASTER_MIN_ZOOM (13) into the publishing
config, and mark the spec execution plan complete. Env-drift check green.
@prbatero
Anthony Cintron (prbatero) force-pushed the prbatero/feat/data-publishing-pc-explorer branch from a6eec4c to 2e6ae14 Compare August 29, 2026 15:11
@github-actions

Copy link
Copy Markdown

RC artifacts ready

All branch deployment references use the same RC tag:

  • hastegeo_version: 1.0.39rc2
  • training_image_tag: 1.0.39rc2
  • imageprep_image_tag: 1.0.39rc2
  • wheel: hastegeo-1.0.39rc2-py3-none-any.whl

Expose the Explorer-render toggle as a first-class deploy input across
both paths (functions.bicep + main.bicep/param, deploy_apps.sh, and the
deploy-apps workflow), mirroring PC_PROVIDER_ENABLED. Defaults on. The
advanced raster knobs stay code-defaults (rarely tuned). Env-drift check
green; bicep builds.
@github-actions

Copy link
Copy Markdown

RC artifacts ready

All branch deployment references use the same RC tag:

  • hastegeo_version: 1.0.39rc3
  • training_image_tag: 1.0.39rc3
  • imageprep_image_tag: 1.0.39rc3
  • wheel: hastegeo-1.0.39rc3-py3-none-any.whl

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds Explorer visualization to the Planetary Computer publishing flow introduced by #123.

Changes:

  • Generates and publishes a damage-classification COG.
  • Registers Explorer render, mosaic, and tile configurations.
  • Adds deployment settings, tests, and design documentation.

Reviewed changes

Copilot reviewed 13 out of 13 changed files in this pull request and generated 7 comments.

Show a summary per file
File Description
spec/features/data-publishing/explorer-visualization.md Documents the visualization design.
infra/modules/functions.bicep Adds the Function App feature flag.
infra/main.bicepparam Reads the deployment flag.
infra/main.bicep Passes the flag to Functions.
hastelib/tests/core/publishing/test_raster.py Tests COG generation.
hastelib/tests/core/publishing/test_planetary_computer_transport.py Tests configuration requests.
hastelib/tests/core/publishing/test_planetary_computer_provider.py Tests asset and configuration integration.
hastelib/src/hastegeo/core/publishing/raster.py Rasterizes damage classifications into a COG.
hastelib/src/hastegeo/core/publishing/planetary_computer_transport.py Adds Explorer configuration operations.
hastelib/src/hastegeo/core/publishing/planetary_computer_provider.py Integrates COG staging and Explorer setup.
hastelib/src/hastegeo/core/config.py Adds visualization settings.
.github/workflows/deploy-apps.yml Forwards the deployment flag.
.github/scripts/deploy_apps.sh Applies the Function App setting.

💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread hastelib/src/hastegeo/core/publishing/planetary_computer_provider.py Outdated
Comment thread hastelib/src/hastegeo/core/publishing/planetary_computer_transport.py Outdated
Comment thread hastelib/src/hastegeo/core/publishing/planetary_computer_transport.py Outdated
Comment thread hastelib/src/hastegeo/core/config.py Outdated
Comment thread hastelib/tests/core/publishing/test_raster.py
Comment thread spec/features/data-publishing/explorer-visualization.md
- raster/provider: only produce the damage_class COG when a predicted-damage
  geopackage is present. A footprints-only publish has no damage column, so
  rasterizing it would publish a misleading all-undamaged layer -- skip both
  the rasterize and the reuse-of-staged-COG paths.
- finalize_unpublish: always delete the dataset-specific published/<id>/ prefix
  instead of gating on the current feature flags. If rendering was enabled at
  publish and disabled before unpublish, the staged COG would otherwise leak.
- transport: accept HTTP 409 as idempotent success when creating render options
  and mosaics, so a concurrent publish racing the get-then-create doesn't abort
  before mosaic/tile-settings are registered.
- config: add _get_bounded_float_env and bound PUBLISH_DAMAGE_RASTER_METERS to a
  positive finite float, so an operator typo fails fast with a clear message
  rather than crashing provider construction or raster generation.
- tests: assert the COG's projected CRS, resolution, and AOI-derived bounds;
  cover the footprints-only skip, the always-clean unpublish, and 409 tolerance.
- spec: add a Contents list to explorer-visualization.md.
@github-actions

Copy link
Copy Markdown

RC artifacts ready

All branch deployment references use the same RC tag:

  • hastegeo_version: 1.0.39rc4
  • training_image_tag: 1.0.39rc4
  • imageprep_image_tag: 1.0.39rc4
  • wheel: hastegeo-1.0.39rc4-py3-none-any.whl

Comment thread .github/scripts/deploy_apps.sh Outdated
Comment thread hastelib/src/hastegeo/core/publishing/raster.py
Comment thread spec/features/data-publishing/explorer-visualization.md
Comment thread hastelib/src/hastegeo/core/publishing/raster.py Outdated
- raster: replace the sorted(zip(...)) with an explicit two-pass partition
  (undamaged then damaged). O(n) instead of O(n log n), no intermediate
  sorted copy, and avoids relying on a sort key to keep non-orderable
  shapely geometries from being compared. damaged_count now counts the
  geometries actually burned (skips empty ones). (mgmachado)
- provider: soften the _stage_damage_class_asset docstring so the
  best-effort guarantee isn't read as unconditional -- an OOM-kill
  terminates the worker rather than raising, so it can still fail the
  publish; the per-side pixel cap keeps that path unlikely. (mgmachado)
- spec: add a Resource use section documenting the raster memory bound
  (~400 MB at the 20000-px ceiling vs a 4096 MB instance) and that
  vector-side memory is uncapped and scales with building count. (mgmachado)
- deploy_apps.sh: drop the lone inline comment on the publishing env block
  to match the surrounding params. (calebrob6)
@github-actions

github-actions Bot commented Sep 1, 2026

Copy link
Copy Markdown

RC artifacts ready

All branch deployment references use the same RC tag:

  • hastegeo_version: 1.0.39rc17
  • training_image_tag: 1.0.39rc17
  • imageprep_image_tag: 1.0.39rc17
  • wheel: hastegeo-1.0.39rc17-py3-none-any.whl

@prbatero
Anthony Cintron (prbatero) merged commit a6e4c93 into main Sep 1, 2026
13 checks passed
@prbatero
Anthony Cintron (prbatero) deleted the prbatero/feat/data-publishing-pc-explorer branch September 1, 2026 12:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature]: Data Publishing & Published Datasets

5 participants